Document
Comment 1： MooreData represents the automated data engineering idea of the MooreData data engineering platform.
Comment 2： Please contact
business@abaka.aifor the latest updates on the platform.Comment 3: To experience our platform, you can log into MooreData.
1. Why was MooreData Platform created?
When it comes to data annotation tools, you may think of open source software such as labelme or labelImg. Open source annotation software is sufficient to meet the data processing needs of 1.0 era (workload of 10,000).
1.1 From Level 1 to Level 2
With the deepening of the neural network, the AI model needs more data to improve the model training effect (workload of 100,000) , and the data annotation has entered the 2.0 era. The original stand-alone annotation tools can no longer meet the needs of this era. So the data annotation platform came out. Through the process-based platform operation, the collaboration of the data production link is realized, and the support for the crowdsourcing model enables elastic expansion of data production capacity.
1.2 From Level 2 to Level 3
With the in-depth implementation of AI in vertical industries, the magnitude of data processing is even larger, often on the order of 100,000 or even 1 million, and the 3.0 era has gradually begun. The data in this era is massive, and the requirements for data accuracy are also higher. It is difficult to meet the needs of this era by relying only on manual labeling and review. Therefore, the human-machine collaborative data production platform based on AI-assisted systems has meet the needs of this era.
In the process of providing high-quality data services for enterprises and institutions in the AI field for a long time, we have combined years of experience in data project management to integrate a series of best solutions into the management module of the platform. At the same time, we invented a more efficient data annotation tool. In order to pursue the ultimate labeling efficiency, we have also integrated AI algorithms into the annotation tools in some scenarios, so that machines and labelers can collaborate to complete data annotation. It also meets the needs of the data platform in the 3.0 era.
Next, let's take a look at the useful functions of ABAVA platform ; )
——Jack Lin
2. Functions introduction
2.1 Data annotation module
We provide processing tools for common data types: image annotation tools, point cloud annotation tools, text annotation tools, audio annotation tools, multimodal annotation tools.
2.1.1 Image annotation tools
Including Image classifacation, Keypoint annotation, 2D line annotation, 2D box annotation, 2D semantic segmentation, Cuboid annotation, OCR annotation, Multi-frame annotation, etc.
2.1.2 Point cloud annotation tools
Including 3D box annotation , 3D semantic segmentation , 3D lane line annotation, 2D/3D fusion annotation, etc.
2.1.3 Text annotation tools
Including NER named entity recognition, SPO text triplet annotation, etc.
2.1.4 Audio annotation tools
Including ASR annotation, Phoneme annotation, etc.
2.1.5 ACL annotation tools
We supports 4D annotation, ACL multimode annotation,etc. In addition, we also supports customized development of other annotation tools. If you have any needs, you can contact business@molardata.comfor specific consultation.
2.1.6 RLHF annotation tools
2.2 Platform management module
2.2.1 2.The best solution for data production
Delivery is the keyword of data production.
In the optimal data production solution of ABAKA AI, we regard the batch as the smallest unit of data delivery. In a data task, each data batch follows the same process for data production.
Taking the following data production process as an example, we have created a 4-node process. Node 1 carries out the labeling process, and nodes 2, 3, and 4 are all review processes. Only one team participates in each node. We will assign node 2 to the supplier team to complete the self-inspection, and node 3 will be assigned to our self-built quality inspection team to complete. Finally in node 4 , we will let the customer do the final quality inspection.
For example, this process is like reimbursement process in the company. The first node is for you to submit the reimbursement application. The second node is for your superior to review, if it is not passed, it will be called back to you for modification, if it is passed, it will enter the third node. The third node is for the review by the manager , if it is not passed, it will be called back to you for modification, if it is passed, it will enter the fourth node. The fourth node is the final node, if it is passed, your reimbursement will come down, if it is not passed, it will be called back and start the process again.
You can choose a process template for data production according to your actual needs, such as 3 nodes (corresponding to double audits) or 5 nodes (corresponding to four audits).
After each node ends, the system will automatically count and analyze the behavior data on the node. As the manager of this data task, we can visually see the sampling accuracy of each node. In addition, we can also know many secrets of the data batch by viewing the details, which is of great significance for subsequent data production. For example, in the batch report, we can know which labeler has the lowest labeling accuracy, then by canceling the labeler's permission to continue annotation, the overall quality of the data can be improved.
2.2.2 Platform management submodule
The platform management function of ABAKA AI includes 3 major sub-modules: task management function module, team management function module, statistical analysis function module.
2.3 AIPower function module
On MooreData data management platform, there is a powerful system that can improve the efficiency of annotation and review, that is, the AIPower closed-loop system. "Closed loop" is a system that can be self-iterative and strengthened.
The AIPower module can provide AI auxiliary functions including AI intelligent labeling and AI intelligent review:
- AI annotation：Before the labeling starts, the algorithm completes the algorithmic reasoning process on a whole batch of data, and obtains the pre-labeling results of a whole batch of data.
- AI audit： After the labeling is completed, the algorithm completes the algorithmic reasoning process for a whole batch of data, compares the actual labeling results with the algorithm review results, and feeds back data with a high possibility of errors.
The AIPower module can also support the iteration of the algorithm model with newly obtained datasets, and provide more accurate AI annotation and AI audit capabilities in the next batch of data production process.
2.3.1 AI annotation function
- AI pre-labelling：Before the labeling starts, the algorithmic system completes the algorithmic reasoning process of the whole batch of data, and obtains the pre-labeling results of the whole batch of data. On this basis, the annotators only make manual revisions and fine-tuning, which can increase the efficiency by 2-10 times.
- AI co-labelling：During the labeling process, the labeler completes a small amount of labeling, and then the algorithm completes the remaining labeling work for this piece of data, and the labeling efficiency can be increased by 2-5 times.
2.3.2 AI review function
After the labeling finishes, the algorithmic system completes the algorithmic reasoning process of the whole batch of data, and compare the actual labeling results with the algorithm review results, then feedback the data with a high possibility of errors for review by reviewers, which will help reduce the error rate by more than 20%.
2.3.2.1 How to manage data program from 0 to 1 ?
The scientific data production process design of MooreData platform allows a user who has no experience in data project management to easily manage and achieve high-quality data production.
2.3.2.2 How to get our data services and tools ?
Our self-developed MooreData Platform can be provided through SaaS and privatization deployment.
Regarding privatization deployment, you can leave your contact information to business@abaka.ai , and we will have a dedicated data expert to provide you with a free 1-to-1 consultation.
In addition, we can also provide you with ACE services.
